Parallel computing

It is a special type of programming that allows the simultaneous execution of several computing task.

Es un tipo especial de programación que permite la ejecución simultánea de varias tareas de cómputo.

Tip

When writing multi-thread programs, you must remember that reading a variable from different threads is not a problem. However, modifying a variable from one thread, and reading its value from another thread requires special attention to avoid data corruption. You must try

Reducing the number of variables that are shared among threads
Use interlocked function instead of critical sections
Use critical sections instead of: mutexes, semaphores or wait functions (::WaitForSingleObjet and ::WaitForMultipleObjects)

Cuando se escriben programa multi-hilo, usted debe recordar que leer una variable desde diferentes threads no es un problema. Sin embargo, modificar una variable desde una thread y leer su valor desde otra thread requiere de atención especial para evitar el daño de los datos. Usted debe tratar

Reducir el número de variables que son compartidas entre threads
Usar las funciones interlocked en lugar de las secciones críticas
Usar las secciones críticas en lugar de: mutexes, semáforos o funciones wait (::WaitForSingleObjet y::WaitForMultipleObjects

::WaitForMultipleObjects

This function is used to wait for several threads in pool to complete. The example below shows how to wait for the three threads to complete.

Esta función es usada para esperar a que varias threas completen. El ejemplo de abajo muestra cómo esperar a que completen las tres threads.

Program.h

#pragma once //______________________________________ Program.h
#include "resource.h"

class Program: public Win::Dialog
{
public:
     Program()
     {
          hA = NULL;
     }
     ~Program()
     {
     }
     HANDLE hA;
     static unsigned WINAPI FunctionA(LPVOID param);
     static unsigned WINAPI Function1(LPVOID param);
     static unsigned WINAPI Function2(LPVOID param);
     static unsigned WINAPI Function3(LPVOID param);
     . . .
};

Program.cpp

. . .
void Program::Window_Open(Win::Event& e)
{
     hA = (HANDLE)::_beginthreadex(NULL, 0, FunctionA, (LPVOID)*this, 0, NULL);
}

unsigned WINAPI Program::FunctionA(LPVOID param)
{
     Program* p = (Program*)param;
     HANDLE h[3];
     h[0] = (HANDLE)::_beginthreadex(NULL, 0, Function1, (LPVOID)p, 0, NULL);
     h[1] = (HANDLE)::_beginthreadex(NULL, 0, Function2, (LPVOID)p, 0, NULL);
     h[2] = (HANDLE)::_beginthreadex(NULL, 0, Function3, (LPVOID)p, 0, NULL);
     // Do other computations . . .

     //______________________________________ 2. Wait for all threads to complete
     ::WaitForMultipleObjects(3, h, TRUE, INFINITY);
     ::CloseHandle(h[0]);
     ::CloseHandle(h[1]);
     ::CloseHandle(h[2]);
     //______________________________________ 3. Notify main thread
     ::PostMessage(p->hWnd, WM_APP, 0, 0);
}

void Program::Window_App(Win::Event& e)
{
     ::CloseHandle(hA);
}

unsigned WINAPI Program::Function1(LPVOID param)
{
     return 0;
}

unsigned WINAPI Program::Function2(LPVOID param)
{
     return 0;
}

unsigned WINAPI Program::Function3(LPVOID param)
{
     return 0;
}

Wait for one object at a time

In some cases, it is possible to incorporate the results from one thread while still waiting for other threads to complete. The following code illustrates how to wait for one thread to complete at a time.

En algunos casos, es posible incorporar los resultados de una thread mientras aún se espera a que otras threads terminen. El código siguiente ilustra cómo esperar por a que una thread complete a la vez.

Program.h

Program.cpp

. . .
void Program::Window_Open(Win::Event& e)
{
     hA = (HANDLE)::_beginthreadex(NULL, 0, FunctionA, (LPVOID)*this, 0, NULL);
}

unsigned WINAPI Program::FunctionA(LPVOID param)
{
     Program* p = (Program*)param;
     HANDLE h[3];
     h[0] = (HANDLE)::_beginthreadex(NULL, 0, Function1, (LPVOID)p, 0, NULL);
     h[1] = (HANDLE)::_beginthreadex(NULL, 0, Function2, (LPVOID)p, 0, NULL);
     h[2] = (HANDLE)::_beginthreadex(NULL, 0, Function3, (LPVOID)p, 0, NULL);
     // Do other computations . . .

     //______________________________________ 2. Wait for one thread at a time
     DWORD wait_count = 3;
     DWORD result;
     int index;
     while (wait_count > 0)
     {
          result = ::WaitForMultipleObjects(wait_count, h, FALSE, INFINITE);
          //if (result == WAIT_TIMEOUT) break;
          //if (result == WAIT_FAILED) break;
          index = result - WAIT_OBJECT_0;
          ::CloseHandle(h[index]);
          //____________________________________3. Incorporate results from h[index] thread
          . . .
          //____________________________________ 4. Update h
          h[index] = h[wait_count-1];
          wait_count--;
     }
     //______________________________________ 4. Notify main thread
     ::PostMessage(p->hWnd, WM_APP, 0, 0);
}

void Program::Window_App(Win::Event& e)
{
     ::CloseHandle(hA);
}

unsigned WINAPI Program::Function1(LPVOID param)
{
     return 0;
}

unsigned WINAPI Program::Function2(LPVOID param)
{
     return 0;
}

unsigned WINAPI Program::Function3(LPVOID param)
{
     return 0;
}

Problem 1

Create a Wintempla Dialog application called MatProd to multiply two matrices using several threads. Each thread requests the next element (row and column using a critical section) to calculate in the resulting matrix. When the element is ready, the thread uses a critical section to write the result. The loop ends when there are no more elements to compute. Observe that results may not be completed in the same sequential order they were initiated.

Cree una aplicación de Dialogo usando Wintempla llamada MatProd para multiplicar dos matrices usando varias threads. Cada thread solicita el próximo elemento (renglón y columna usando una sección crítica) a calcular en la matriz resultante. Cuando el elemento está listo, la thread usa una sección crítica para escribir el resultado. El ciclo while termina cuando no hay más elementos a calcular. Observe que los resultados pueden no ser completados en el mismo orden secuencial en el que fueron iniciados.

Step A

Use Wintempla to activate the App event (WM_APP) as shown. This event is used to notify to the main thread that one of the worker threads has ended.

Use Wintempla para activar el evento App (WM_APP) como se muestra. Este evento es usado para notificar a la thread principal que una de las threads trabajadoras ha terminado.

Step B

Edit the MatProd.h and MatProd.cpp files as shown. Do not forget to implement the Mt::IThreadX interface.

Edite los archivos MatProd.h y MatProd.cpp como se muestra. No se olvide de implementar la interface Mt:IThreadX.

MatProd.h

#pragma once //______________________________________ MatProd.h
#include "Resource.h"
#define WORK_ID 1
#define NUM_THREADS 4

class MatProd: public Win::Dialog, public Mt::IThreadX
{
public:
     MatProd()
     {
          rowIndex = 0;
          colIndex = 0;
           rowCountA = 0;
          colCountA = 0;
          rowCountB = 0;
          colCountB = 0;
          numThreads = 0;
     }
     ~MatProd()
     {
     }
     Sys::Stopwatch sw;
     int numThreads;
     long rowIndex;
     long colIndex;
     long rowCountA;
     long colCountA;
     long rowCountB;
     long colCountB;
     MATRIX matrixA;
     MATRIX matrixB;
     MATRIX matrixResult;
     Mt::CriticalSection csResult;
     Mt::CriticalSection csPosition;
     Mt::ThreadObjectX thread[NUM_THREADS];
     //____________________________________________________________ IThreadX
      DWORD ThreadFunc(Mt::BoolTs& cancel, size_t threadIndex, size_t numThreads);
     . . .
};

MatProd.cpp

...
void MatProd::Window_Open(Win::Event& e)
{
     long i, j;
     //____________________________________________________________ 1. Create: matrixA, matrixB and matrixResult
     Math::Oper::CreateMatrix(matrixA, 2000, 800);
     Math::Oper::CreateMatrix(matrixB, 800, 300);
     Math::Oper::CreateMatrix(matrixResult, 2000, 300);
     rowCountA = (long)matrixA.size();
     colCountA = (long)matrixA[0].size();
     rowCountB =(long)matrixB.size();
     colCountB = (long)matrixB[0].size();
     //____________________________________________________________ 2. Initialize A with random values
     for (i = 0; i < rowCountA; i++)
     {
          for (j = 0; j < colCountA; j++)
          {
               matrixA[i][j] = rand()/(double)RAND_MAX;
          }
     }
     //____________________________________________________________ 3. Initialize B with random values
     for (i = 0; i < rowCountB; i++)
     {
          for (j = 0; j < colCountB; j++)
          {
               matrixB[i][j] = rand()/(double)RAND_MAX;
          }
     }
     sw.Start();
     //____________________________________________________________ 2. Start Threads
     for (int i = 0; i < NUM_THREADS; i++)
     {
          thread[i].threadIndex = i;
          thread[i].numThreads = NUM_THREADS;
          thread[i].BeginThread(*this);
          numThreads++;
     }
     wchar_t text[64];
     _snwprintf_s(text, 64, _TRUNCATE, L"Threads: %d", numThreads);
     this->Text = text;
}

DWORD MatProd::ThreadFunc(Mt::BoolTs& cancel, size_t threadIndex, size_t numThreads)
{
     long result_row = 0;
     long result_col = 0;
     double result = 0.0;
     long k = 0;
     while(true)
     {
          //_________________________________________________________________ 1. Compute: result_row and result_col
          csPosition.Enter();
          //_____________________________________________ 1.1. Are we done?
          if (rowIndex >= rowCountA)
          {
               csPosition.Leave();
               break;
          }
          //_____________________________________________ 1.2. Get: result_row and result_col
          result_row = rowIndex;
          result_col = colIndex;
          //_____________________________________________ 1.3. Move to next position in matrixResult (update rowIndex and colIndex)
          colIndex++;
          if (colIndex >= colCountB)
          {
               rowIndex++;
               colIndex = 0;
          }
          csPosition.Leave();
          //____________________________________________________________________2. Compute result
          result = 0.0;
          for (k = 0; k < colCountA; k++)
          {
               result += matrixA[result_row][k] * matrixB[k][result_col];
          }
          //__________________________________________________________________ 3. Copy result to matrixResult[result_row][result_col]
          csResult.Enter();
          matrixResult[result_row][result_col] = result;
          csResult.Leave();
     }
     //____________________________________________________________________ 4. Report completion to main thread
     ::PostMessage(hWnd, WM_APP, (WPARAM)threadIndex, (LPARAM)WORK_ID);
     return 0;
}

void MatProd::Window_App(Win::Event& e)
{
     if (e.lParam == (LPARAM)WORK_ID)
     {
          const int threadIndex = (int)e.wParam;
          if (threadIndex < 0 || NUM_THREADS <= threadIndex) return;
          numThreads--;
          thread[threadIndex].WaitForExit();
          if (numThreads == 0)
          {
               this->Text = sw.GetMillisecondsText();
               // Sys::FileAssistant::CsvSave(L"C:\\selo\\MatProd.csv", matrixResult);
          }
          else
          {
               wchar_t text[64];
               _snwprintf_s(text, 64, _TRUNCATE, L"Threads: %d", numThreads);
               this->Text = text;
          }
     }
}

Step C

Edit the number of threads at the top of the MatProd.h file and complete the following table writing down the execution time for each case. To complete the last column, compile the program from Microsoft Visual Studio using the Release configuration, and then double click the MatProd.exe file directly from the Release folder using Windows Explorer.

Edite el número de threads en la parte superior del archivo MatProd.h y complete la tabla siguiente escribiendo el tiempo de ejecución para cada caso. Para completar la última columna, compile el programa desde Microsoft Visual Studio usando la configuración de Release, y entonces haga doble clic en el archivo MatProd.exe directamente desde la carpeta de Release usado el Windows Explorer.

Number of Threads	Debug	Release	Release (outside Visual Studio)
1
2
3
4
5
6
7
8
9
10

Step D

Edit the number of threads at the top of the MatProd.h file and use the Windows Task Manager to complete the following table with the CPU usage.

Edite el número de threads en la parte superior del archivo MatProd.h y use el Administrador de Tareas para completar la siguiente tabla con el Uso del CPU.

Number of Threads	Debug	Release	Release (outside Visual Studio)
1
2
3
4
5
6
7
8
9
10

Tip

The following code illustrates how to adjust the number of threads based on the number of CPUs.

El código siguiente ilustra cómo ajustar el número de threads basado en el número de CPUs.

MathProd.h

#pragma once //______________________________________ MatProd.h
#include "Resource.h"
#define WORK_ID 1

class MatProd: public Win::Dialog, public Mt::IThreadX
{
public:
     MatProd()
     {
          SYSTEM_INFO sysinfo;
          ::GetSystemInfo(&sysinfo);
          numProcessors = sysinfo.dwNumberOfProcessors;
          thread = new Mt::ThreadObjectX[numProcessors];
          ...
     }
     ~MatProd()
     {
          if (thread != NULL) delete [] thread;
     }
     int numProcessors;
     Mt::ThreadObjectX *thread
     ...
};

Problem 2

When several threads try to enter a critical section or exchange information, it is possible that performance gets affected. A good parallel program must be designed so that each thread performs a long task and only occasionally enter a critical section or other synchronization method to share data. Create a Wintempla Dialog application called MatProdX to multiply two matrices using several threads. In this case, each thread will compute a complete row of the resulting matrix instead of one element at a time. Observe that this program does not need a critical section to get the next element position. Do not forget to implement the Mt::IThreadX interface. Do not forget to check the WM_APP event in the main window.

Cuando varias threads tratan de en entrar una sección crítica o intercambian información, es posible que el desempeño resulte afectado. Un buen programa en paralelo debe ser diseñado para que cada thread realice una tarea larga y solamente ocasionalmente entre en una sección crítica u otro medio de sincronización para compartir datos. Cree una aplicación de Dialogo usando Wintempla llamada MatProdS para multiplicar dos matrices usando varias threads. En este caso, cada thread calculará un renglón completo de la matriz resultante en lugar de un elemento a la vez. Observe que este programa no necesita una sección crítica para obtener la posición del siguiente elemento. No se olvide de implementar la interface Mt:IThreadX. No se olvide de seleccionar el evento WM_APP en la ventana principal.

MatProdX.h

#pragma once //______________________________________ MatProdX.h
#include "Resource.h"
#define WORK_ID 1

class MatProdX: public Win::Dialog, public Mt::IThreadX
{
public:
     MatProdX()
     {
          rowIndex = -1;
          rowCountA = 0;
          colCountA = 0;
          rowCountB = 0;
          colCountB = 0;
          numThreads = 0;
          //
          SYSTEM_INFO sysinfo;
          ::GetSystemInfo(&sysinfo);
          numProcessors = sysinfo.dwNumberOfProcessors;
          thread = new Mt::ThreadObjectX[numProcessors];

     }
     ~MatProdX()
     {
          if (thread != NULL) delete[] thread;
     }
     Sys::Stopwatch sw;
     int numThreads;
     volatile long rowIndex;
     long rowCountA;
     long colCountA;
     long rowCountB;
     long colCountB;
     MATRIX matrixA;
     MATRIX matrixB;
     MATRIX matrixResult;
     Mt::CriticalSection csResult;
     //
     int numProcessors;
     Mt::ThreadObjectX *thread;
     //____________________________________________________________ IThreadX
     DWORD ThreadFunc(Mt::BoolTs& cancel, size_t threadIndex, size_t numThreads);
     . . .
};

MatProdX.cpp

. . .
void MatProdX::Window_Open(Win::Event& e)
{
     long i, j;
     //____________________________________________________________ 1. Create: matrixA, matrixB and matrixResult
     Math::Oper::CreateMatrix(matrixA, 2000, 800);
     Math::Oper::CreateMatrix(matrixB, 800, 300);
     Math::Oper::CreateMatrix(matrixResult, 2000, 300);
     rowCountA = (long)matrixA.size();
     colCountA = (long)matrixA[0].size();
     rowCountB =(long)matrixB.size();
     colCountB = (long)matrixB[0].size();
     //____________________________________________________________ 2. Initialize A with random values
     for (i = 0; i < rowCountA; i++)
     {
          for (j = 0; j < colCountA; j++)
          {
               matrixA[i][j] = rand()/(double)RAND_MAX;
          }
     }
     //____________________________________________________________ 3. Initialize B with random values
     for (i = 0; i < rowCountB; i++)
     {
          for (j = 0; j < colCountB; j++)
          {
               matrixB[i][j] = rand()/(double)RAND_MAX;
          }
     }
     sw.Start();
     //____________________________________________________________ 2. Start Threads
     for (int i = 0; i < numProcessors; i++)
     {
          thread[i].threadIndex = i;
          thread[i].numThreads = numProcessors;
          thread[i].BeginThread(*this);
          numThreads++;
     }
     wchar_t text[64];
     _snwprintf_s(text, 64, _TRUNCATE, L"Threads: %d", numThreads);
     this->Text = text;
}

DWORD MatProdX::ThreadFunc(Mt::BoolTs& cancel, size_t threadIndex, size_t numThreads)
{
     long result_row = 0;
     long col = 0;
     double result = 0.0;
     long k = 0;
     while (true)
     {
          //_____________________________________________ 1. Get: result_row
          result_row = ::InterlockedIncrement(&rowIndex);
          //_____________________________________________ 2. Are we done?
          if (rowIndex >= rowCountA) break;
          //_____________________________________________ 3. Compute one row of resulting matrix
          for (col = 0; col < colCountB; col++)
          {
               result = 0.0;
               for (k = 0; k < colCountA; k++)
               {
                    result += matrixA[result_row][k] * matrixB[k][col];
               }
               //____________________________________________ 4. Copy result to matrixResult[result_row][col]
               csResult.Enter();
               matrixResult[result_row][col] = result;
               csResult.Leave();
          }
          if (cancel == true) break;
     }
     //____________________________________________________________________ 4. Report completion to main thread
     ::PostMessage(hWnd, WM_APP, (WPARAM)threadIndex, (LPARAM)WORK_ID);
     return 0;
}

void MatProdX::Window_App(Win::Event& e)
{
     if (e.lParam == (LPARAM)WORK_ID)
     {
          const int threadIndex = (int)e.wParam;
          // Sys::PrintLine(L"%d / %d", threadIndex, numThreads);
          if (threadIndex < 0 || numProcessors <= threadIndex) return;
          numThreads--;
          thread[threadIndex].WaitForExit();
          if (numThreads == 0)
          {
               this->Text = sw.GetMillisecondsText();
               //Sys::FileAssistant::CsvSave(L"C:\\selo\\MatProdX.csv", matrixResult);
          }
          else
          {
               wchar_t text[64];
               _snwprintf_s(text, 64, _TRUNCATE, L"Threads: %d", numThreads);
               this->Text = text;
          }
     }
}

Step A

Measure the duration of the program.

Mida la duración del programa.

Number of Threads	Debug	Release	Release (outside Visual Studio)
1
2
3
4
5
6
7
8
9
10

Step B

Measure the CPU usage.

Mida el uso del CPU.

Number of Threads	Debug	Release	Release (outside Visual Studio)
1
2
3
4
5
6
7
8
9
10

Problem 3

Create a Wintempla Window application called Rain to display the results of a background computation. After creating the application, open Wintempla, double click on the editor and check the events: App and Timer.

Cree una aplicación de Ventana usando Wintempla llamada Rain para mostrar los resultados de un cálculo en segundo plano. Después de crear el programa, abra Wintempla, haga clic doble en el editor y selecciones los eventos: App y Timer.

Step A

From Microsoft Visual Studio menu select Tools > Add Wintempla Item... > TableView to insert a TableView. Set the name to CircleView and press Add.

Desde el menú de Microsoft Visual Studio seleccione Tools > Add Wintempla Item... > TableView para insertar una TableView. Fije el nombre a CircleView y presione Add.

Step B

Open the Rain.h file and use Wintempla to insert a custom control as shown. Be sure the Show All Controls in Toolbox in selected.

Step C

From Wintempla Menu use the Dock button to dock the custom control to the four borders of the window as shown.

Desde el menú de Wintempla use botón de Dock para sujetar el control personalizado a los cuatro bordes de la ventana cómo sé muestra.

Step D

Edit the Rain.h and Rain.cpp files as shown. Be sure to implement the Mt::IThread interface as shown.

Edite los archivos Rain.h y Rain.cpp cómo sé muestra. Asegúrese de implementar la interface Mt::IThread cómo se muestra.

Rain.h

#pragma once //______________________________________ Rain.h
#include "Resource.h"
#include "CircleView.h"
#define WORK_ID 100

class Rain: public Win::Window, public Mt::IThread
{
public:
     Rain()
     {
     }
     ~Rain()
     {
     }
     CircleView td;
     Mt::ThreadObject threadObject;
     DWORD ThreadFunc(Mt::BoolTs& cancel, Mt::DecimalTs& progress, Mt::BoolTs& resetTime);
     const wchar_t * GetClassName(){return L"Rain";}
protected:
     ...
};

Rain.cpp

#include "stdafx.h" //________________________________________ Rain.cpp
#include "Rain.h"

...

void Rain::Window_Open(Win::Event& e)
{
     this->Text = L"Working ...";
     this->timer.Set(0, 1000);
     threadObject.BeginThread(*this);
}

DWORD Rain::ThreadFunc(Mt::BoolTs& cancel, Mt::DecimalTs& progress, Mt::BoolTs& resetTime)
{
     const size_t count = 99999999;
     const size_t maxRadio = MINIMUM(customControlCircleView.ColumnWidth, customControlCircleView.RowHeight);
     customControlCircleView.circle.resize(count);
     for (size_t i = 0; i < count; i++)
     {
          progress = 100.0*i/count;
          if (cancel == true) break;
          customControlCircleView.circle[i] = (maxRadio*rand())/RAND_MAX;
     }
     ::PostMessage(hWnd, WM_APP, (WPARAM)0, (LPARAM)WORK_ID);
     return 0;
}

void Rain::Window_App(Win::Event& e)
{
     if (e.lParam == WORK_ID)
     {
          threadObject.WaitForExit();
          timer.Kill(0);
          this->Text = L"Rain";
          customControlCircleView.ReportImageProcessing();
     }
}

void Rain::Window_Timer(Win::Event& e)
{
     if (e.wParam != 0) return;
     wchar_t text[64];
     threadObject.GetProgressInfo(text, 64);
     this->Text = text;
}

Step E

Edit the CircleView.h and CircleView.cpp files as shown.

Edite los archivos CircleView.h y CircleView.cpp cómo sé muestra.

CircleView.h

#pragma once //_____________________________________________ CircleView.h
//To create an object of this class, you must insert a Custom Control in the GUI
class CircleView : public Win::ScrollControl
{
public:
     CircleView();
     ~CircleView();
     void ReportImageProcessing();
     vector<size_t> circle;
     Mt::BoolTs imgReady;
     //___________________________________ It is called when painting each cell column
     void ScrollPaint(int paintEvent, CG::Gdi& gdi, const RECT& cell, int rowIndex, int colIndex, bool isSelected);
     void OnFontSet(const LOGFONT& logFont);
};

CircleView.cpp

#include "stdafx.h"
#include "CircleView.h"

CircleView::CircleView()
{
     RowHeight = 60;
     ColumnWidth = 80;
     imgReady = false;
}

CircleView::~CircleView()
{
}

void CircleView::ReportImageProcessing()
{
     imgReady = true;
     ColumnCount = width/ColumnWidth;
     RowCount = (int)ceil(circle.size()/(double)ColumnCount);
}

void CircleView::ScrollPaint(int paintEvent, CG::Gdi& gdi, const RECT& cell, int row, int col, bool isSelected)
{
     if (paintEvent == WNT_PAINTCELLSBEGIN)
     {
          gdi.SetBkMode(true);
     }
     else if (paintEvent == WNT_PAINTCELL)
     {
          if (imgReady == false) return;
          const size_t size = circle.size();
          if (size == 0) return;
          const size_t index = row*this->ColumnCount+col;
          if (index >= size) return;

          int x = (cell.right + cell.left)/2;
          int y = (cell.top + cell.bottom)/2;
          gdi.Circle(x, y, circle[index]/2);
     }
     else if (paintEvent == WNT_PAINTCELLSEND)
     {
          gdi.SetBkMode(false);
     }
}

void CircleView::OnFontSet(const LOGFONT& logFont)
{
}

Problem 4

Create a Win32 console application called ParallelDot to compute the dot product between two vectors using C++ multithreading.

Cree una aplicación de consola de Win32 llamada ParallelDot para calcular el producto punto entre dos vectores usando C++ multithreading.

ParallelDot.cpp

// ParallelDot.cpp : This file contains the 'main' function. Program execution begins and ends there.
//

#include "pch.h" // #include "stdafx.h"
#include <iostream>
#include <thread>

void DotProduct(float* a, float*b, size_t first, size_t last, float& result)
{
     result = 0.0;
     for (size_t i = first; i < last; i++) result += a[i]*b[i];
}

int main()
{
     //_______________________________________ 1. Memory allocation
     const size_t len = 65536;
     float* a = new float[len];
     float* b = new float[len];
     //_______________________________________ 2. Set a and b with random values from -500.0 to 500.0
     size_t i;
     for (i = 0; i < len; i++)
     {
          a[i] = (1000.0f*rand())/RAND_MAX - 500.0f;
          b[i] = (1000.0f*rand())/RAND_MAX - 500.0f;
     }
     //_______________________________________ 3. Create an array of threads and start each thread
     const size_t numThreads = std::thread::hardware_concurrency();
     std::thread* threadPool = new std::thread[numThreads];
     float* result = new float[numThreads];
     size_t delta = len/numThreads;
     size_t first, last;
     for ( i = 0; i < numThreads; i++)
     {
          first = i*delta;
          last = (i == numThreads -1) ? len-1 : first+delta;
          std::cout<<"Starting thread "<<i+1<<" computing from "<<first<<" to "<<last-1<<"\r\n";
          threadPool[i] = std::thread(DotProduct, a, b, first, last, std::ref(result[i]));
     }
     //_______________________________________ 4. Wait for each thread to complete and consolidate result
     float product = 0.0;
     for ( i = 0; i < numThreads; i++)
     {
          threadPool[i].join();
          product += result[i];
     }
std::cout<<"Parallel Dot product = "<<product<<"\r\n";
     //_______________________________________ 5. Use regular dot product computation
     float regular = 0.0f;
     DotProduct(a, b, 0, len, regular);
     std::cout<<"Regular Dot product = "<<regular<<"\r\n";
     //_______________________________________ 6. Clean up
     if (a != NULL) delete [] a;
     if (b != NULL) delete [] b;
     if (threadPool != NULL) delete [] threadPool;
     if (result != NULL) delete [] result;
}

MSDOS: cmd.exe

Starting thread 1 computing from 0 to 8191
Starting thread 2 computing from 8192 to 16383
Starting thread 3 computing from 16384 to 24575
Starting thread 4 computing from 24576 to 32767
Starting thread 5 computing from 32768 to 40959
Starting thread 6 computing from 40960 to 49151
Starting thread 7 computing from 49152 to 57343
Starting thread 8 computing from 57344 to 65534
Parallel Dot product = -1.01793e+07
Regular Dot product = -1.01528e+07

C:\selo\ParallelDot\Debug\ParallelDot.exe (process 3464) exited with code 0.
Press any key to close this window . . .

Parallel computing